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ABSTRACT 

Participant observation is a research and evaluation 
strategy in dealing with complex educational settings, organizations, 
and curricula is explored. Three alternative models are presented* 

The first is a general structural model involving three research 
strategies: an experimental design with pretests and posttests; a 
social survey with interviews and questionnaires; and, a participant 
observer study of the program. Second, a sequential model which 
attempts to cumulate efforts over time rather than concurrently in 
time. r j his model is illustrated in some detail by use of a study cn 
teacher awareness. The third strategy of participant observation and 
curriculum evaluation is the cumulation of case studies. 

Substrategies under this model are suggested. (LR) 
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lntroduction 

In the general abstract for todays presentation we described 
our intentions in this way: 

The symposium addresses the special methodological problems 
in the utilization of participant observation in curriculum evaluation. 
While the techniques have been tried in a number of settings this 
presentation concentrates on the analysis of the Ber 1 ak-Toml i nson 
Elementary Social Studies Curriculum. The application and integration of 
the point of view should be of use to curriculum developers, imp letnen ters 
and evaluators . " 

My responsibilities are twofold: to introduce you to the 

participants and to make a few substantive comments of my own . The 
Introductions Ci handled best perhaps by relating briefly some of 
the history of the presentations. Essentially it's a story of the 
weaving of two relatively independent strands of activity into a common 
experience. 

One of the most exciting aspects of Washington University's 
Graduate Institute of Education over the last few years has been a 
vigorous thrust In social studies curriculum and instruction. With their 
roots In the 01 1 ver-Shave r tradition at Harvard, Harold Berlak and Tim 
Tomlinson have sought to develop a series of elementary school curri- 
culum units which would involve kth , 5th, and 6th grade pupils In 
experiences of Mexican, Russian, Indian, I ta 1 1 an-Ame r 1 can and other 
children who are facing difficult decisions In their day to day lives. 

The problems of peer group living, family stresses and strains, and 
neighborhood changes contain basic dilemmas In value choices. A variety 
of experiences - textual material, slide tapes, role playing, group 
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discussions are used to engage the children. As the curriculum was 
being developed, concerns for evaluation - formative and sunmative were 
raised. Skepticism had long been felt for the limitations of some 
of the more usual research and evaluation strategies. 

Concurrently with these developments, another strand of 
activity at Washington University and also CEMREL, Inc., our nearby 
regional laboratory, was the exploration of participant observation 
as a research and evaluation strategy for approaching complex educa- 
tional settings, organizations, and curricula. Early efforts had 
focused on a middle class teacher coping with a class of lower class 
children, on the origins and development of an innovative elementary 
school, on an unusual teacher training program, on implementing a 
computer assisted instruction program in a rural community and in a 
structured discovery general science curriculum In the high school. 

The two strands ran together in a curriculum evaluation 
seminar, which might best be described as a free-for-all. Parentheti- 
cally, unfortunately I might say, one of the things the new curriculum 
doesn't do is to teach people to respect their elders. Consequently, 
Harold Berlak and I barely survived. Applegate, Self, and Solomon 
pursued problems in evaluating the Ber 1 ak-Toml I nson curriculum; some 
of these they will be discussing today.* 



* We owe thanks to Ann Berlak, George Talrgrleve, John Good, anti Paul 
Pohland who pursued other problems, but whose ideas and points of view 
enlivened and facilitated the discussions. 
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Integrating Participant Observation Into Broader Evaluation Strategies 

At this point I want to shift from moderator to panelist 
and develop briefly an argument regarding the relationship of parti- 
cipant observation to more general research and evaluation strategies. 
Three alternatives seem plausible and reflect our efforts so far: 

1) a general structural model^ 2) a sequential model^and 3) a case 
study accumulation model. 

A General Structural Mode? 

My own thinking and use of participant observation had been 
initially as a general research strategy to approach complex naturalist- 
ic problems. Howard Russell, then at fEMREL, and f‘ began talking about 
a "three legged 11 model of evaluation, The model sought to bring three 
research strategies: 1) an experimental design with pre and post tests 

of achievement, control groups, and inferential statistics. 2) A 
second strategy was the social survey with Interviews and ques t iona i res , 
random sampling of program relevant individuals (teachers, parents, 
and pupiVs), quantification, and cross tabulation of response . 3)The 
third "leg" of the model was a participant observer study of the pro- 
gram. 

My colleague Paul Pohland and I engaged in the description 
and analysis of the mundane day to day operation of the program (Smith & 
Pohland, I9&9i 1971) • We have observed children at the teletypes, 
talked with teachers about the joys arid tribulations, and inquired 
Into arithmetic instruction. As It turned out, the problems in keeping 
the program running were severe. This moved us into a careful and 
serious consideration of the problems in putting highly sophisticated 
21st Century Technology Into an Impoverished rural area of the nation. 
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Our data analysis suggested issues in complex technical and social systems. 
The roots of the innovation problems were as varied and complex as 
congressional funding patterns (lateness, cuts, rerouting through 
the state department) and the mobilization of five independent tele- 
phone companies to install lines and equipment. V/e think we have 
Important data for understanding this kind of curriculum change. 

The major point 1 would make is not that people have not 

i 

used direct observation in curriculum development and evaluation, but 

i 

that they have not exploited i t as a major tool In the analysis. In 

if 

ij 

a recent AERA curriculum evaluation monograph, Grobman (1968) devotes 
several pages to what she calls "visits/ 1 Her introductory paragraph 
states : 

No project can afford to omit classroom visits, and 
such visits can serve a variety of purposes. Visits 
can serve to verify other feedback or to put it In a 
more meaningful context. Teachers who are reluctant to 
write criticism or who find writing difficult may talk 
quite openly in a face-to-face encounter. Conversation 
with school officials, teachers, students, and parents 
can elicit Information that cannot be provided by 
questionnaires and may open up new avenues of thought 
not previously considered by the project. [p. 5*0 

In effect, we have taken seriously her doubts regarding the validity 
of responses people give. More basically though, we see the parti- 
cipant observer strand as an attempt to describe and conceptualize 
the nature of a very complex independent variable-- the nature and 
utilization of the new curriculum. Such a research strategy has a 
potency which we felt had not been utilized fully before. 

As Russel! (19 69 ) pointed out In the Introductory chapter 
of the final report In the CAI evaluation the combined analysis made 
a powerful summative analysis of the program. The learning gains, the 

0 
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rlse and fall of changing attitudes toward the program, and the im- 
plementation of the program by pupils, teachers, classes, and organ- 
izations was clarified. 

The Sequential Kodal 

The sequential model attempts to cumulate efforts over time 
rather than concurrently in time. The major position we take is that 
the kind of field work we have been doing is important for the gen- 
eration rather than the verification of hypotheses. To accomplish 
the latter, one moves to other research paradigms. For instance, in 
our Intensive observational study of an urban classroom (Smith £ 
Geoffrey, 1 968) we utilized the concept of ’’teacher awareness" to 
interpret some of our data. The concept was defined as: 

a dimension of teacher behavior in which the teacher 
knows information important in the group members' lives 
and indicates his knowledge to the group. [p. ^70] 

One of the explicit illustrations used to educe the concept was the 
teacher's teasing of an adolescent boy about, his girl -friend and about 
the fact that he, the teacher, might have to move their seats. Be- 
sides the two adolescents, at least one audience pupil had an incredu- 
lous look on her face, A second illustration involved a pupil's seeming 
intention to "fool" the teacher in getting an extra turn at a simple 
and pleasurable alphabetizing task. The teacher caught her at the game 
and she responded with a sheepish grin and a return to her seatwork. 

We developed a number of hypotheses surrounding the phenomenon of 
teacher awareness. 

Paul Kleine and I sought to explore the Implications of 
the concept teacher awareness (Smith £ Kleine, 19&9). In an Inten- 
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slve theoretical analysis of the concept of cognitive complexity as 
this has grown out of the Kelly tradition and cognitive differentiation 
from the Witkin tradition, Kleine ( 1 968 ) argued for the theoretical 
similarity of the ideas. Each is concerned with the degree of struc- 
ture (differentiation or complexity) the individual possesses in his 
conceptual organization of the environment. We predicted that these 
personality variables would correlate with teacher awareness and would 
be important antecedents of this part of the ongoing classroom sit- 
uation. That is, the teachers with the more differentiated and com- 
plex cognitive structures would be more aware, more knowledgeable of 
the ongoing classroom social system. In our analysis we hypothesized 
also that teacher awareness, the knowledge of events in p up i I s r lives, 
leads to pupil esteem for the teacher. Pupil esteem refers to the 
generalized sentiment which the pupils hold for the teacher. For many 
years commentators have talked about pupil attitudes toward school, toward 
lessons, and toward their teachers. Some Investigators of attitude 
learning and opinion change have argued that prestige and esteem are 
Important social psychological variables. 

While the conception of teacher awareness began with several 
Intriguing illustrative observations from our field study (Smith & 
Geoffrey, 1968), the translation we made for quantitative purposes 
proceeded as follows; 

(1) each teacher rank ordered her pupils on three dimensions: 
popularity, arithmetic ability, and psychomotor ability; 

(2) the pupils In each class filled out a best friends choice 
type socloinetric questionnaire regarding their classmates. 
They took a short arithmetic achievement test. And they 

• 7 



-7- 



f 1 1 led out a "Guess who?" type soc'ometrlc perception 
questionnaire regarding psychomotor ability; 

(3) correlations between teacher rankings and pupil measures 
were obtained for each classroom; 

(4) the correlation coefficients were converted to z scores 
and combined to form a single score of teacher awareness. 

Methodologically, a sample of 69 teachers and their classes 
was drawn from the CEMP.EL region (Tennessee, Kentucky, Missouri, and 
Illinois), Some were from rural and small town common j t i es ; others 
were suburban. The majority of teachers were female (58). Thei; 
range of experience varied from one to forty-nine years and averaged 
sixteen years. All classes were at the fifth and sixth grade levels. 

Among the results we found a significant correlation between 
cognitive complexity and teacher awareness and a signif ? ;ant corre- 
lation between teacher awareness and pupil esteem.* 

However, my point is not theoretical but methodological. 

V7e have found the field study important for the generation of con- 
cepts, hypotheses, and miniature theories. These ideas can then be 
operationalized, quantified, and tested in broad-scale correlational 
analyses as we did with "teacher awareness." Hopefully also, these 
ideas can be moved into even irore rigorous experimental designs. Only 
after that kind of endeavor can one have confidence that the findings 
pertain to more than our one case. The sequential blending of the 
techniques seems to produce extra benefits. 



-Correlational data have known and admitted limitations for 
cause-effect interpretations. 

ERJT 
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Cumulating Case Studies 

A third strategy of participant observation and curriculum 
evaluation is the one we are illustrating today; the cumulation of 
participant observation case studies. In general, the argument we have 
been making is that participant observation is especially fruitful 
for what Glaser & Straus (1967) have called the "generation of grounded 
theory." However, as one engages in multiple case studies, questions 
arise regarding the increasing credibility of hypotheses and models 
generated in one setting and now reappearing in second and third 
settings. In turn, we have been asking ourselves additional questions: 
1) Must we always turn to the quantitative experimental model for 
verification? 2) Are there some problems that the laboratory cannot 
rep 1 i ca te--as Kounin (1970) suggests in his recent work on classroom 
management? 3) Are research skills different enough and idiosyncratic 
enough that specialization of labor is a major issue? *0 Are enough 
persons doing enough work with any one technique to be especially 
facile with it? And so forth. 

Within the cumulative case study approach several sub 
strategies seem to viable: First, a sequential thrust of one class- 

room study, then another, then another. In ef fee t^ cumu la t i ng cases 
(descriptions, concepts, hypotheses and models) in the same domain. 
Second, 3 sequential thrust wherein one moves from classrooms to 
curricula to school to organ i zat ion to community analysis. In effect, 
one maps enlarging and Interlocking domains into more comprehensive 
theory. Third, and more illustrative of our efforts In today's 
symposium is the concurrent attack upon several parts of the same 
curriculum. For instance a concern for the curri cu lum wr i ter ' s In- 
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te:itlons compared and contrasted with the realities of the situation, 
a focus on the teacher's behavior and the kind of thought processes 
generated by the curriculum, and reconccptua I i zat ion of a special 
technique - role playing which Is a major tactic in ;he curriculum. 

Such a concurrent "t r i angu I a ted T ' attack seems to have a further kind 
of analytical po vier. 

Cone 1 us ion 

In this symposium we are raising issues regarding partici- 
pant observation as a research and evaluation strategy. Individually 
and collectively the members of this symposium have been actively 
involved in the approach and its problems and its relationships to other 
points of view. The three most general W3ys of integrating parti- 
cipation into more general research and evaluation models might be 
labeled I) the general structural model 2) the sequential model 
3) the case study accummulat ion model. Each brings certain pluses 
as veil as certain weaknesses. Each holds promise for making increas- 
ingly credible statements about important problems In curriculum and 
I ns t ruct i on . 
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